design and validation
Design and Validation of a Responsible Artificial Intelligence-based System for the Referral of Diabetic Retinopathy Patients
Moya-Sánchez, E. Ulises, Sánchez-Perez, Abraham, Da Veiga, Raúl Nanclares, Zarate-Macías, Alejandro, Villareal, Edgar, Sánchez-Montes, Alejandro, Jauregui-Ulloa, Edtna, Moreno, Héctor, Cortés, Ulises
Diabetic Retinopathy (DR) is a leading cause of vision loss in working-age individuals. Early detection of DR can reduce the risk of vision loss by up to 95%, but a shortage of retinologists and challenges in timely examination complicate detection. Artificial Intelligence (AI) models using retinal fundus photographs (RFPs) offer a promising solution. However, adoption in clinical settings is hindered by low-quality data and biases that may lead AI systems to learn unintended features. To address these challenges, we developed RAIS-DR, a Responsible AI System for DR screening that incorporates ethical principles across the AI lifecycle. RAIS-DR integrates efficient convolutional models for preprocessing, quality assessment, and three specialized DR classification models. We evaluated RAIS-DR against the FDA-approved EyeArt system on a local dataset of 1,046 patients, unseen by both systems. RAIS-DR demonstrated significant improvements, with F1 scores increasing by 5-12%, accuracy by 6-19%, and specificity by 10-20%. Additionally, fairness metrics such as Disparate Impact and Equal Opportunity Difference indicated equitable performance across demographic subgroups, underscoring RAIS-DR's potential to reduce healthcare disparities. These results highlight RAIS-DR as a robust and ethically aligned solution for DR screening in clinical settings. The code, weights of RAIS-DR are available at https://gitlab.com/inteligencia-gubernamental-jalisco/jalisco-retinopathy with RAIL.
A Nested Model for AI Design and Validation
Dubey, Akshat, Yang, Zewen, Hattab, Georges
The growing AI field faces trust, transparency, fairness, and discrimination challenges. Despite the need for new regulations, there is a mismatch between regulatory science and AI, preventing a consistent framework. A five-layer nested model for AI design and validation aims to address these issues and streamline AI application design and validation, improving fairness, trust, and AI adoption. This model aligns with regulations, addresses AI practitioner's daily challenges, and offers prescriptive guidance for determining appropriate evaluation approaches by identifying unique validity threats. We have three recommendations motivated by this model: authors should distinguish between layers when claiming contributions to clarify the specific areas in which the contribution is made and to avoid confusion, authors should explicitly state upstream assumptions to ensure that the context and limitations of their AI system are clearly understood, AI venues should promote thorough testing and validation of AI systems and their compliance with regulatory requirements.
The LHCb ultra-fast simulation option, Lamarr: design and validation
Anderlini, Lucio, Barbetti, Matteo, Capelli, Simone, Corti, Gloria, Davis, Adam, Derkach, Denis, Kazeev, Nikita, Maevskiy, Artem, Martinelli, Maurizio, Mokonenko, Sergei, Siddi, Benedetto Gianluca, Xu, Zehua
Detailed detector simulation is the major consumer of CPU resources at LHCb, having used more than 90% of the total computing budget during Run 2 of the Large Hadron Collider at CERN. As data is collected by the upgraded LHCb detector during Run 3 of the LHC, larger requests for simulated data samples are necessary, and will far exceed the pledged resources of the experiment, even with existing fast simulation options. An evolution of technologies and techniques to produce simulated samples is mandatory to meet the upcoming needs of analysis to interpret signal versus background and measure efficiencies. In this context, we propose Lamarr, a Gaudi-based framework designed to offer the fastest solution for the simulation of the LHCb detector. Lamarr consists of a pipeline of modules parameterizing both the detector response and the reconstruction algorithms of the LHCb experiment. Most of the parameterizations are made of Deep Generative Models and Gradient Boosted Decision Trees trained on simulated samples or alternatively, where possible, on real data. Embedding Lamarr in the general LHCb Gauss Simulation framework allows combining its execution with any of the available generators in a seamless way. Lamarr has been validated by comparing key reconstructed quantities with Detailed Simulation. Good agreement of the simulated distributions is obtained with two-order-of-magnitude speed-up of the simulation phase.
Design and Validation of a Wireless Drone Docking Station
Stuhne, Dario, Vasiljevic, Goran, Bogdan, Stjepan, Kovacic, Zdenko
Drones are increasingly operating autonomously, and the need for extending drone power autonomy is rapidly increasing. One of the most promising solutions to extend drone power autonomy is the use of docking stations to support both landing and recharging of the drone. To this end, we introduce a novel wireless drone docking station with three commercial wireless charging modules. We have developed two independent units, both in mechanical and electrical aspects: the energy transmitting unit and the energy receiving unit. We have also studied the efficiency of wireless power transfer and demonstrated the advantages of connecting three receiver modules connected in series and parallel. We have achieved maximum output power of 96.5 W with a power transfer efficiency of 56.6% for the series connection of coils. Finally, we implemented the system in practice on a drone and tested both energy transfer and landing.
Design and Validation of a Multi-Arm Relocatable Manipulator for Space Applications
Hoffman, Enrico Mingo, Laurenzi, Arturo, Ruscelli, Francesco, Rossini, Luca, Baccelliere, Lorenzo, Antonucci, Davide, Margan, Alessio, Guria, Paolo, Migliorini, Marco, Cordasco, Stefano, Raiola, Gennaro, Muratore, Luca, Rodrigo, Joaquín Estremera, Rusconi, Andrea, Sangiovanni, Guido, Tsagarakis, Nikos G.
This work presents the computational design and validation of the Multi-Arm Relocatable Manipulator (MARM), a three-limb robot for space applications, with particular reference to the MIRROR (i.e., the Multi-arm Installation Robot for Readying ORUs and Reflectors) use-case scenario as proposed by the European Space Agency. A holistic computational design and validation pipeline is proposed, with the aim of comparing different limb designs, as well as ensuring that valid limb candidates enable MARM to perform the complex loco-manipulation tasks required. Motivated by the task complexity in terms of kinematic reachability, (self)-collision avoidance, contact wrench limits, and motor torque limits affecting Earth experiments, this work leverages on multiple state-of-art planning and control approaches to aid the robot design and validation. These include sampling-based planning on manifolds, non-linear trajectory optimization, and quadratic programs for inverse dynamics computations with constraints. Finally, we present the attained MARM design and conduct preliminary tests for hardware validation through a set of lab experiments.
Design and Validation of a Portable Machine Learning-Based Electronic Nose
Volatile organic compounds (VOCs) are chemicals emitted by various groups, such as foods, bacteria, and plants. While there are specific pathways and biological features significantly related to such VOCs, detection of these is achieved mostly by human odor testing or high-end methods such as gas chromatography–mass spectrometry that can analyze the gaseous component. However, odor characterization can be quite helpful in the rapid classification of some samples in sufficient concentrations. Lower-cost metal-oxide gas sensors have the potential to allow the same type of detection with less training required. Here, we report a portable, battery-powered electronic nose system that utilizes multiple metal-oxide gas sensors and machine learning algorithms to detect and classify VOCs. An in-house circuit was designed with ten metal-oxide sensors and voltage dividers; an STM32 microcontroller was used for data acquisition with 12-bit analog-to-digital conversion. For classification of target samples, a supervised machine learning algorithm such as support vector machine (SVM) was applied to classify the VOCs based on the measurement results. The coefficient of variation (standard deviation divided by mean) of 8 of the 10 sensors stayed below 10%, indicating the excellent repeatability of these sensors. As a proof of concept, four different types of wine samples and three different oil samples were classified, and the training model reported 100% and 98% accuracy based on the confusion matrix analysis, respectively. When the trained model was challenged against new sets of data, sensitivity and specificity of 98.5% and 98.6% were achieved for the wine test and 96.3% and 93.3% for the oil test, respectively, when the SVM classifier was used. These results suggest that the metal-oxide sensors are suitable for usage in food authentication applications.
Design and validation of world-class multilayered thermal emitter using machine learning
NIMS, the University of Tokyo, Niigata University and RIKEN have jointly designed a multilayered metamaterial that realizes ultra-narrowband wavelength-selective thermal emission by combining the machine learning (Bayesian optimization) and thermal emission properties calculations (electromagnetic calculation). The joint team then experimentally fabricated the designed metamaterial and verified the performance. These results may facilitate the development of highly efficient energy devices. Thermal radiation, a phenomenon that an object emits heat as electromagnetic waves, is potentially applicable to a variety of energy devices, such as wavelength-selective heaters, infrared sensors and thermophotovoltaic generators. Highly efficient thermal emitters need to exhibit emission spectrum with narrow bands in practically usable wavelength range..